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QUESTIONS FOR THE COMMITTEE 


In cases where estimates of variance are not available, does the Committee have 
advice on whether variance should be approximated proportional to 1/cell 
magnitude, or as 1/magnitude*? (Section 4.1) 


Is it appropriate to use maximum likelihood estimation (MLE) methods to 
determine weighting parameters, as a substitute for subjective ratings of data 
accuracy? (Section 4) 


Is it appropriate to use a MLE-oriented approach for balancing tables with a time 
series component, as an alternative to previously-published methods based on 
adding movement- and level-preservation objective functions? (Section 5) 


Is the method described in Section 5.2 appropriate for reducing bias in forward- 
series estimates e.g. for quarterly National Accounts benchmarking? 


Are the diagnostics identified for balancing appropriate, and are there others 
that should be considered? Is it possible to adapt leverage-type diagnostics for 
this application? (Section 6) 


Are approximation/iteration approaches preferable to penalty function methods 
for dealing with nonlinear constraints/non-quadratic objective functions? 
(Section 7) 


Are the proposed strategies for handling large problems appropriate, including 
rolling estimates subject to revision? (Section 8) 


Does the Committee have any other advice on the methods discussed here? 
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ABSTRACT 


National Accounts and other statistical outputs frequently require “balancing” or 
“benchmarking”: adjusting estimates to satisfy internal consistency constraints and/or 
to reconcile information from multiple sources. Mathematical optimisation 
techniques such as weighted least squares balancing (WLS) are used for this purpose 
by other agencies and offer advantages over current ABS methods. The additional 
flexibility of WLS methods will improve the production and quality of National 
Accounts estimates, by making better use of data quality information and saving time 


currently spent in manual processes. 


Ideally, weighting of adjustments would be based on the variance of the unbalanced 
estimate. When variance data is unavailable (e.g. non-survey sources), a fall-back 
method is required. This paper offers arguments for fall-back weighting according to 
the magnitude of unadjusted value (as opposed to squared magnitude). 


The literature on WLS balancing has often approached these problems with the goal 
of minimising changes to levels and/ or changes to time series movements. This 
paper considers the problem of producing a maximum likelihood estimator (MLE) for 
the true data values, and shows that for certain important cases, WLS movement- or 
level-preservation is in fact equivalent to a MLE-based solution. It also offers a 
method, usable in both MLE- and movement-preservation frameworks, for reducing 
bias in forward-series estimates for balanced time series. 


WLS balancing requires setting weighting parameters that indicate expected error 
behaviour. This has generally been done through numerical models which require 
parameters to be set subjectively, e.g. through estimation by subject matter experts. 
This paper suggests an alternative way for setting parameters: by framing balancing as 
a MLE problem, a MLE approach can be applied to estimating these parameters, 
reducing subjectivity and need for manual involvement in the process. 


This paper then extends the MLE approach to give a WLS method covering cases 
where both movements and levels are of interest, and shows that a previously- 
published WLS method for handling such cases is not equivalent to the MLE solution. 


This paper also discusses options for balancing diagnostics, for handling nonlinear 
constraints with software that only supports linear constraints, and for dealing with 
very large problems whose size may present computational difficulties. 
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1. INTRODUCTION 


The Australian Bureau of Statistics has recently started a program of infrastructure 
renewal. This program includes updates to a range of ABS systems and processes, 
offering an opportunity for re-evaluation and improvement of these processes. 


Several of these processes can be framed as numerical optimisation problems, e.g. 
making the most cost-effective use of ABS field interviewers, creating an efficient 
sample design that meets multiple objectives, or reconciling inconsistent data sources 
to create high quality estimates that combine the strengths of those sources. 


The availability of high quality commercial and open source optimisation software on 
an off-the-shelf basis presents an opportunity for ABS to improve these processes 
while reducing maintenance and support burdens, and where possible bringing 
different applications into the same conceptual framework. 


This paper concentrates on National Accounts applications, in particular balancing 
economic activity tables, because improving efficiency and output quality for 
economic statistics is a priority in ABS’ current transformation plan. However, many 
of the issues discussed here will also be relevant to other ABS applications. 


Major objectives of our work are: 


: Increase versatility of National Accounts balancing systems, reducing the need 
for manual balancing and time taken to balance tables. 


° Improve quality of outputs by applying consistent framework for initial balancing 
of tables, and making better use of knowledge about data sources. 


° Share knowledge with other statistical institutions using similar methods and 
improve understanding of methods used by these institutions. 


The success of optimisation methods depends on being able to design an “objective 
function” that adequately quantifies one’s goals. This paper offers a theoretical basis 
for doing so, by reviewing currently-used implementations of optimisation and 
reinterpreting them as maximum likelihood estimators (MLEs). 


This MLE interpretation offers a theoretical justification for some currently-used 
methods, and offers a more objective way to resolve some of the decisions made in 
their implementation e.g. setting weights for objective functions. Section 2 gives the 
context for the National Accounts balancing problem, and presents a simple example. 


Section 3 presents a theoretical framework for basic balancing problems, showing an 
equivalence between WLS level-preservation and maximum likelihood estimation, and 
identifies limitations and assumptions of that framework. 
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Section 4 discusses methods used to set weights for WLS level preservation, and offers 
a MLE-based approach to reduce the need for subjective choices in this process. 


Section 5 extends the MLE approach to time series considerations, in particular 
movement-preservation adjustments. It shows that simple movement -preservation 


objective functions can also be interpreted as MLEs. 


Despite the MLE basis for level- and movement-preservation objective functions when 
taken individually, adding these objectives together to form a combined objective 
function cannot be easily justified from a MLE perspective. However, this paper 
identifies another option: adding the error models implicit in those objectives to 
create a combined error model, then constructing a new MLE/WLS objective function 
appropriate to that model. 


Sections 6, 7, and 8 discuss related issues: diagnostics for optimisation balancing, 
methods for dealing with nonlinear constraints, and options for scaling methods to 
very large problems. 


Methods used for this work should be defensible, minimising the use of subjective 
decision making. The complexity of solutions should be appropriate for maintenance 
by staff without a specialist background in optimisation. 
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2. CONCEPTUAL BACKGROUND 


This section discusses the background to National Accounts balancing work, with 


further theoretical detail to be provided in subsequent sections. 


2.1 Motivation: need for balancing in National Accounts 


An important part of National Accounts production is compiling tables of balanced 


estimates for various kinds of economic activity. The relationships between these 


items of interest imply certain rules about how their values should relate to one 


another. 


For example, table 2.1 shows an estimated breakdown of the value of goods and 


services supplied in a year by sectors of a fictional economy.’ Note that some 


industries may be associated with more than one product, and vice versa: e.g. a 


business whose primary activity is telecommunications may also provide financial 


services as a secondary activity. 


2.1 Supply of goods and services 


Industry 
event lron Ore Steel Car Tele Fin Oth Other Total 
Mining Mfg Mfg Svcs Svcs Svcs} Imports Supply Supply 
Iron Ore 400 0) 0) ) O ) 0) 50 450 
Steel 50 500 0) ) 0) 0) 160 100 810 
8 Cars ) ) 700 ) ) 0} 1,000 300 2,000 
s Tele Svcs 0) ) 0) 400 50 20 150 80 700 
Fin Svcs ) 0) 0) 20 1,100 50 200 150 1,520 
Oth Svcs 10 0) 100 ) 80 120 50 40 400 
Total Output 460 500 800 420 1,230 190 1,560 720 5,880 


Table 2.2 shows the value of goods and services used by these same sectors, along 


with gross value added by each industry (GVA) and changes to inventories. 


1 


4 


In practice, accounts tables are combined not only in values but also in volumes, and values may be assessed 
under more than one set of prices (see e.g. Nicolardi, 2011). For simplicity this discussion assumes that tables 
are to be balanced in values; volumes and alternate prices can be incorporated through ratio constraints. 
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2.2 Use of goods and services 


Industry 
USE Iron 

Ore Steel Car Tele Fin Oth Invent- Other Total 
Mining Mfg Mfg Svcs Svcs Sves| Exports __ ories Use Use 
Iron Ore 20 250 0) ) ) ) 190 40 O 500 
Steel ) 0) 350 ) 0) 70 150 30 0) 600 
8 Cars ) ) ) 0) 0) 0) 300 -100 1,800 2,000 
& Tele Svcs 100 120 150 200 80 50 50 ) 250 1,000 
Fin Svcs 50 60 110 180 450 20 100 0) 500 1,470 
Oth Svcs 30 40 30 40 150 10 20 0) 180 500 
Total Use 200 470 640 420 680 150 810 -30 2,730 6,070 

GVA 245 33 154 10 540 35 

Total Input 445 503 794 430 1,220 185 


Together, these estimates of supply and use constitute a Supply-Use (“SU”) table. 
Economic definitions impose several constraints on the values for these items, e.g.: 
° For any product in the economy, total supply must match (“balance”) total use. 
° For any industry in the economy, total input must match total output. 


. Most items must be non-negative. Exceptions exist e.g. GVA and changes to 
inventories. 


: Some items are conceptually zero, e.g. inventories for services; these are called 
“structural zeroes”. 


° Many other items are assumed small enough that they can be approximated as 
zero, even if non-zero values are possible (e.g. supply of wool by mining 
businesses); these are also considered structural zeroes. 


Note that in the example shown above, the requirements on total supply/total use and 
total input/total output are vot satisfied: e.g. total supply of steel is 810 but total use is 
only 600. 


This example has been simplified for convenience. A real SU table for Australia 
contains 301 products and 67 industries, along with some extra terms e.g. taxes, 
subsidies, private financial and non-financial, household and government sectors, but 
the same kinds of requirements apply. In some cases it may be desirable to balance 
several years of data simultaneously; this will be discussed in more detail in Section 6. 


Data for such tables are compiled from a variety of sources including government 
taxation records, ABS industry surveys, and import/export declarations. Many of these 
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sources are subject to some degree of error, e.g. due to misreporting or sample-based 
estimation, and these errors lead to inconsistency within tables. 


While some error in these tables is inevitable, inconsistency can create serious 
problems for analysis (as per the example, starting from the premise that 6(00=810, 
literally anything can be proven!) 


Inconsistency also implies that the tables could be made more accurate through 
better use of the available data. Referring to the example above, the discrepancy 
between supply and use of steel indicates that some of the use estimates may be too 
low, and hence overall accuracy could be improved by increasing them, or alternately 
that some supply estimates are too high and should be reduced. 


Major discrepancies may be best resolved by subject matter experts who can 
investigate causes and estimate an appropriate correction, but the size of these tables 
makes it time consuming to do all balancing through manual processes. This is 
further complicated by the multi-dimensional nature of such tables: an adjustment 
that balances a row may unbalance a column, and the balancing must reconcile three 
different measures of gross domestic product. 


These factors create a requirement for an automated balancing process (“auto- 
balancing”) that adjusts table entries to satisfy applicable constraints. 


Usually there will be many different ways to balance the table. In the example above 
we could increase estimates for steel use items, decrease estimates for steel supply, or 
some combination of the two, so that total supply aligns with total use. We could 
even satisfy the constraints listed above by adjusting all values to zero, but this would 
not be a realistic solution. 


The object then becomes to find a solution that balances the table, with the least 
possible disruption to the data overall; later sections will discuss how exactly to 
quantify overall disruption. 


These tables are typically compiled from multiple sources that may differ greatly in 
their accuracy. For example, data on tax receipts can be obtained directly from 
taxation records with very high accuracy; imports and exports are also available from 
government data. Other items may be somewhat less accurate, having been obtained 
from sample surveys or similar sources. Furthermore, since most industries use a 
wide range of products but supply only a few, data on the Supply side will often be 
more accurate than that on the Use side. 


A good balancing process would factor in these differences in accuracy, using 
weighting or other methods to encourage smaller adjustments for items known to be 
reliable. 


6 ABS * QUADRATIC OPTIMISATION FOR TABLE BALANCING IN OFFICIAL STATISTICS * 1352.0.55.157 


ABS METHODOLOGY ADVISORY COMMITTEE * JUNE 2016 


Business needs may impose other requirements on the balancing process. For 
example, there may be a need to maintain consistency with a previous publication; 
e.g. if we have already published an estimate that the total value of financial services 
supplied is $1.25 billion, then we may require that the balanced cells are consistent 
with this total, while allowing individual cells to vary within that. 


2.2 Related problems 


The focus of this paper is on what are considered “balancing” problems, such as the 
provided Supply-Use example. 


However, many of the same concepts and tools can be applied to other problems 
within National Accounts. An important example is the benchmarking/temporal 
disaggregation problem, where we have two sources of data relating to (e.g.) a 
quarterly time series for an industry of interest. The “benchmark” data gives very 
accurate totals at an annual level; the “indicator” series has larger errors but is known 
to correlate with the quarterly movements. These two sources can be combined to 
give an accurate quarterly series, by using annual benchmarks to calibrate the 
quarterly indicators (cf. Dagum and Cholette, 2006; International Monetary Fund, 
2014). 


There are also combination benchmarking/balancing problems, e.g. multivariate 
benchmarking where we seek to estimate multiple time series for various industries, 
requiring consistency with a quarterly-level benchmark across multiple industries as 
well as the annual industry-level benchmarks already discussed. Section 6.3 presents 
an example of such a problem. 


Another case (not further discussed in this paper) is the need to publish estimates for 
supply and use of commodities in both current prices and volumes, with the 
requirement that both versions balance individually while preserving a relationship 
between them. 


These problems can be expressed in the same way as those set out in Section 3: 
continuous optimisation problems with linear constraints and a quadratic objective 
function. 


Similar problems also arise in official statistics outside National Accounts work. For 
example, calculation of Estimated Resident Population also requires adjusting fine 
level population estimates for consistency with known aggregates, and could be 
approached through the same methods. Although this paper focuses on National 
Accounts and specifically on table balancing applications, the principles discussed 
here should also be understood as relevant to those other applications. 
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Other important ABS optimisation problems require choosing solutions from a 
discrete set of options and are better approached as linear integer problems (LIPs), 
e.g. field interviewer allocation where the variables represent yes/no decisions. LIP 
methods are outside the scope of this paper, but in many cases the software tools that 
would be used for quadratic optimisation are also suitable for LIPs, and the theoretical 
frameworks are similar enough that familiarity with one class of problem will be 
beneficial when approaching the other. 


2.3 Approaches to solving balancing problems 


Lenzen et al. (2009) identify two major classes of methods used for balancing 
problems: 


2.3.1 RAS balancing and entropy theory approaches 


0 6b oan ad 


RAS? (aka “iterative proportional fitting”, “matrix raking”, “iterative scaling”, “cross- 
entropy’) relies on iteratively balancing subsets of the problem, or numerically 
equivalent techniques. 


For example, in the balancing problem shown in Section 2.1, unbalanced Total Use for 
iron ore is 500, but Total Supply is 450. Taking the Total Supply data as authoritative, 
we would scale all Use items for iron ore by 450/500 = 0.9, ensuring that the Use total 
matches Supply. We would then repeat the process for other rows, e.g. for steel 
(supply=810, use=600) we multiply all use items by 810/600. 


Next we would use the same process to balance columns. This unbalances rows, but 
the discrepancies will be less than in the original inputs. We then iterate row and 
column scaling until the data converges to a balanced table. 


Currently ABS uses manual adjustment for Supply-Use tables, with RAS then used to 
balance finer level Input-Output data to SU benchmarks. A separate implementation 
of RAS is used elsewhere in ABS for calculation of Estimated Resident Population. 


Naive RAS methods are relatively simple to implement and their computational 
requirements are modest, which can be important when balancing large data sets. 
However, they have some important limitations: 


° Zero values will never be adjusted to non-zero, even when this might be 
appropriate, and sign changes are impossible even when this might be 
appropriate (Lenzen et al., 2014). 


: RAS does not converge when constraints based on primary data are conflicting. 


° Missing values need to be replaced by non-missing before balancing. 


2 Notan acronym. Named for Stone’s notation 7AS ; see Bacharach (1965). 
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° When a row contains both positive and negative values, RAS may produce 
unnecessarily large adjustments. For example, suppose a given item has 
unbalanced use of $110m and change in inventories of -$100m (adding to $10m 
Total Use) but is required to match a figure of $30m Total Supply. In this case 
RAS would triple the Use and Inventories items, resulting in adjustments of 200% 
(before any changes caused by rebalancing columns). A more sensible solution 
might be e.g. to adjust use to $120m and inventories to -$90m. 


. Naive RAS does not consider information about data reliability and cannot 
handle conflicting external data. 


Various refinements have been proposed that address these issues (see e.g. Lenzen 
et al., 2006, 2009, 2012a, 2012b, 2013, 2014). However, these methods increase the 
complexity of a RAS-based solution and are likely to require a custom-built solver. 


Various other approaches to balancing are based on concepts of information- 
theoretical entropy, e.g. minimising information loss as measured by a Kullback- 
Leibler divergence function. McDougall (1999) notes that RAS is equivalent to 
minimisation of cross-entropy, although some authors have not been aware of this 
relationship. 


Detailed discussion of entropy-based approaches is out of scope for this paper. 
Without attempting to compare their theoretical properties to the WLS/MLE approach 
outlined below and in Section 3, it seems likely that WLS/MLE would be significantly 
easier to implement at ABS for practical reasons. In particular, the concepts involved 
in WLS/MLE will be more familiar to ABS staff and more easily explained, and in some 
cases have strong parallels to existing ABS processes. 


2.3.2 Optimisation balancing 


Subject to these limitations, RAS generally achieves balanced outputs with reasonably 
small adjustments, but these may not be the smallest possible. An alternative to RAS- 
type balancing is a more exact optimisation approach: we explicitly define the quality 
of potential solutions as a function of the adjusted values (“objective function”), and 
then use numerical methods to optimise on that objective function’. 


For example, in the Supply-Use scenario presented in Section 2.1 above, we might 
define the objective function as a weighted sum of squares of the adjustments to 
individual cells. Linear algebra methods can then be used to find a solution that 
minimises this sum of squares, subject to the specified constraints. 


3 RAS and entropy methods can also be framed as optimisation, but here I follow Lenzen et al. in making a 
distinction between such methods and those that support a more customisable objective function. 
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Optimisation methods have several strengths: 


Optimisation methods do not assume any particular structure to the data to be 
balanced; rather, structure is specified by individual constraints. This allows 
flexibility in what sort of constraints can be imposed; for example, tables can 
easily be balanced in more than two dimensions, a key requirement for National 
Accounts. 


The objective function can incorporate information about accuracy of 
contributing estimates: e.g. cells known to be accurate can be weighted heavily 
in the objective function, discouraging large adjustments to these cells. Where 
appropriate, values can be fixed to zero adjustment. 


Negative and zero values do not require special treatment. 


Missing values and their weights would ideally be modelled or otherwise 
imputed as per the literature on missing data methods, but in cases where it is 
not possible to do so, they can be accommodated by deleting the corresponding 
terms from the objective function. Under the MLE framework discussed in 
Section 3, this is equivalent to assuming an arbitrarily-chosen unbalanced value 
with an infinitely large variance (i.e. a flat prior). 


Quadratic optimisation problems are important in a wide range of commercial 
applications (e.g. costs in electricity transmission networks can be described by a 
weighted least squares objective function) so many commercial and open-source 
products have been developed for solving them, and a great deal of research has 
gone into developing efficient methods for solving large problems quickly.’ 


The history of optimisation balancing goes back to the description of a weighted least 


squares framework by Stone ef al. (1942). At the time and for many years afterwards, 


lack of computing power made this approach infeasible; Lenzen et al. (2009) note that 


optimisation methods have high computing requirements and programming 


complexity when compared to RAS. 


More recently, improvements in computing power and off-the-shelf availability of 


powerful general purpose constrained optimisation solvers have brought these 


methods within reach. For example, the U.S. Bureau of Economic Analysis now uses 


CPLEX software for reconciliation and balancing work (Rassier et al., 2007a and 2007b) 


and Statistics Netherlands uses Xpress-MP for the same purpose. The Australian 


Bureau of Statistics and other national statistical organisations are currently evaluating 


and developing optimisation methods. 


4 


10 


In the related field of mixed integer programming, Koch et al. (2011) report that between 1990 and 2010, 
algorithm advances gave a 55,000 improvement in solution speed. 
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3. THEORETICAL FRAMEWORK 


This section sets out a framework for mathematical discussion of optimisation 
balancing. 


3.1 Derivation of weighted least squares objective function 


It is well known (see e.g. Bradley, 2009) that weighted least squares estimators for a 
regression model are equivalent to maximum likelihood estimators (“MLEs”) when 
certain conditions are satisfied. A similar result applies in balancing-type problems:° 


Let x be a vector (5%, WaigaXs ): containing the true (possibly unobservable) values 
for quantities to be estimated in a table, with 7 being the length of x. 


Considerations discussed in Section 2 place some constraints on the possible values of 
x. For the time being, we assume that these constraints are of two kinds: 


1. Linear equality constraints e.g. Supply=Use, structural zeroes, or other fixed 
values/subtotals within the table. 


2. Linear inequality constraints e.g. values that cannot be negative. 


These constraints can be collectively expressed in matrix-vector form as: 


where A”’ and A“”’ are known matrices of dimensions Rk”? xn and k’”? xn 


'XF2 are known constant vectors of length k“? and k’“”” 


respectively, and c”° and ¢ 
respectively, with R’° and k’”’’ being the number of equality and inequality 
constraints. The “feasible region” is the set of all possible solutions for x that would 


satisfy these constraints. 


From some source we obtain “unbalanced” estimates x that approximate the true 
series, but with measurement errors: 


KR=X+E 


t ; , ‘ , , 
&=(&,,&,,...,€,) with estimation errors {¢,} being random variables. In general, 


ty 


these errors mean that X will not satisfy the balancing constraints. 


5 Stone et al. (1942) observed that a weighted average of GDP measures satisfies the principle of maximum 
likelihood, but there is little if any mention of this MLE/WLS connection in more recent balancing literature. 
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If we have information about the joint distribution of {e,} , we can use this to produce 


EQ 


a balanced estimate X that is close to x but satisfies the constraints A”°x =c*? and 


A™2%>c"° This can be described as adjusting the unbalanced values, with the 
vector of adjustments® being r =X —% , but it is also useful to think of it in terms of 
producing an estimate of the errors, €=X—X. Within this section r =—&, but in 
some cases (see e.g. Section 6) we may have additional error terms, in which case this 


identity may not hold. 


If we assume that {e,} are independent normally distributed variables having 
distributions ¢, = N (0, a; ) it follows that for any possible choice of €, the prior 
probability density for ¢ = € is equal to: 


exp (-& /207 ) 


f(-=T] pee 


We can then select x through a maximum likelihood estimation approach, choosing 
the value from the feasible region that maximises this probability. 


Since In is a monotonically increasing function, maximising fis equivalent to 
minimising a negative log-transform: 


-in(s(@))=2 Z +Da(o,v27), 


; oO 


The second term in this expression is constant, so this is the same as minimising the 
quadratic function De; i.e. a weighted sum of squared adjustments, where the 
weight w, fora given cell is the reciprocal of its variance: w, =1/07. 


We will refer to functions of this type as objective functions or “OFs”. Later we will 
examine other objective functions and their corresponding error models. For clarity 


this particular function will be referred to as OF 


ry Since its form emphasises 


preservation of levels: 
as a \2 a 2 
OF 4 = > W, (Re -%) <2 R=% +e, 6 =N(0,07), w, = 152 ; 
i 


OF,,, can also be expressed in matrix form: OF,,, = &'Wé where W = diag(w). 


6 Strictly speaking, it would be more accurate to consider these as residuals, to encompass e.g. cases where x 
includes two independent measurements of the same x, . For simplicity the notation here assumes a one-to- 
one correspondence between elements of x and of X , but the method can easily be adapted for other 
possibilities. 
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This formulation has some desirable characteristics: 


It is mathematically convenient. In cases where the inequality constraints are 
ignorable, the problem can be reduced to a closed form matrix inversion. In 
cases where this solution would violate inequality constraints, a solution can be 
obtained by iterative application of closed form methods, usually requiring only 
a few iterations. Solving the problem can be computationally expensive, when a 
large number of variables are involved, but various commercial and free software 
products are designed for this purpose. 


It is easy to interpret and supports fine level examination of balancing results. 
Because the OF breaks down into a sum of terms that each relate to a single 
adjustment, it can be understood as assigning a cost to each individual 
adjustment. These costs can then be used for fine level diagnostics, to be 
discussed in Section 4. 


It is flexible. For example, if the unbalanced estimate for one of the x-variables is 
missing and we have no good basis for imputing it, this can be handled by 
setting the corresponding term in the objective function to zero (equivalent to 
having an infinitely large variance). If multiple independent sources are available 
for the same x-variable, the objective function can include a term for each of 
them. 


Quadratic optimisation problems with linear constraints are a very important 
class of problem in operations research, so powerful tools have been developed 
to solve them. 


Some implications of the assumptions made in the above derivation: 


Assumption of non-biased errors: where possible, known sources of systematic 
error (bias) should be corrected before balancing, to satisfy the assumption that 
errors have expected value zero. Eurostat recommends that even though 
optimisation balancing is able to eliminate large discrepancies, these should 
instead be addressed and resolved by specialists. This becomes particularly 
important when using methods that assume small adjustments relative to initial 
values, as discussed in Sections 6 and 7. 


Assumption of independence: inputs should be examined for relationships likely 
to lead to correlation between error terms, and ramifications of such correlation 
should be considered (see Section 3.2 below). 
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Assumption of normality: in cases where non-normally distributed errors are 
expected, the WLS estimate may not be equivalent toa MLE. This may arise e.g. 
where large outliers make Central Limit Theorem approximations unreliable. A 
least squares framework is mathematically and computationally convenient; 
simulation studies may be desirable in order to assess how sensitive WLS 
balancing methods are to non-normality. 


Flat prior assumption (implicit in the choice of a MLE): in cases where we have 
strong information about expected distributions of true values (other than that 
observed in x ), it may be possible to produce a better anterior estimate by 
modifying the objective function according to that prior. Such methods are 
outside the scope of this paper. 


3.2 Covariance between cells 


The discussion above assumes that {é,} are independent. In some cases this 


assumption does not hold, in particular: 


Tt 


Cases where some dimensions of x are derived from others without being 
measured separately: e.g. the table includes unbalanced estimates for “use of 
steel” by industry, and the “total use of steel” item has been calculated as the 
sum of these other estimates. In such cases, the derived x,and corresponding 
é, will be completely dependent on the dimensions they are derived from. (In 
other words, the mechanism that produces these data automatically enforces an 
equality constraint.) 


Cases where dimensions are measured separately, but those measurements are 
not completely independent, due to effects in the measurement process, leading 
to some correlation in the {é,} . 


Note that a balancing constraint that applies to the true values does not automatically 


create dependence in the unbalanced estimates. For example, we might have some 


data source on total use of steel that is independent of the sources used to compile 


the use-by-industry estimates. 


Here we assume that it is possible to divide an unbalanced table up into two groups of 


cells having the following properties: 


14 


“basic cells”: cells representing (approximately) independent measurements, 
whose ¢€, are hence independent of one another. 


“derived cells”: cells whose unbalanced values and corresponding ¢€, are 
completely determined by the basic cells and their corresponding ¢, . 
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Since the values for basic cells completely determine the values of derived cells, we 


obtain the correct maximum likelihood objective function by weighting basic cells 


according to their variance, and weighting all derived cells at zero. 


If we ignore these dependencies, e.g. by weighting all cells (both basic and derived) at 


1/ variance, the objective function is effectively over-weighting the data that 


contributes to derived cells. This can lead to sub-optimal adjustments, and would 


mean that the results of the balancing can be changed by adding extra sub-total rows 


or columns, even though these contain no new information. 


As a simple illustration, suppose that we expand the Use table presented in 2.1 to 


include a new “manufacturing industries subtotal” column (table 3.1) with unbalanced 


values derived as the sum of unbalanced steel and car manufacturing activity: 


3.1 Supply of goods and services, with added “subtotal” column 


Industry 

SUPPLY Iron Ore Steel Car Mfg Tele Fin Oth Other Total 
Mining Mfg Mfg subtotal Svcs Svcs Sves | Imports Supply Supply 

Iron Ore 400 0 0) 0 0 0) 0) 0 50 450 
Steel 50 500 0) 500 0) 0) 0 160 100 810 

8 | cars 700 700 fo) 0) 0} 1,000 300 2,000 
& Tele Svcs 0) fe) 400 50 20 150 80 700 
Fin Svcs 0) 0 20 1,100 50 200 150 1,520 

Oth Svcs 10 100 100 0) 80 120 50 40 400 
Total Output 460 500 800 1,300 420 1,230 190 1,560 720 5,880 


Previously the objective function included a single term for each item created by 


manufacturing industries. Showing only the terms related to supply for 


manufacturing industries: 


OF, 


upply|Mfg ~~ 2 


S steel |SteeLMFG 


2 


© Cars\CarMFG 


Z B> Sae oe 2 
ie siccies = 500) : Cae ~ 700) e Care Zz 100) 


2 


© oupsves|ca 7+MFG 
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But if we include the “manufacturing industries subtotal” entries in the objective 
function without regard for covariance, this becomes: 


< 2 a 2 
( X seel |SteelIMFG 5 00 ) ( X sieel |MFGSubtotal — 5 00 ) 
OF, = ag 


Suppl |Mfg ~~ 2 2 
9 steel |SteeLMFG O steel |MFGSubtotal 


i 2 2 
(eae = 700) 2 (Sones = 700) 


2 2 
© cars|carMFG © cars| MFGSubtotal 


2 2 
( X oubsves|CarMFG — 100 ) rs ( X oubsvcs|MFGSubtotal a 00) 


2 2 
© ounsves|CarMFG © Ornsucs|MFGSubtotal 


However, the derivation of the subtotal means that x aX 


Steel|SteelMFG Steel|MFGSubtotal ? 


2 vhs ; 
© sieel|steelMFG ms © sveei| MFGSubtotal and so forth. Therefore: 


Se, 2 = 2 Po 2 
(Xe scsiacd ~ 500) 49 (seat =. 700) 42 (cascecrane ~ 100) 


2 
Oca rs|CarMFG 


OF; 


upply|\ Mfg 


2 2 
© steel|steelMFG © otpsves|\CarMFG 
Hence, the effect of including this subtotal in the objective function is equivalent to 
doubling the weights on each item supplied by manufacturing industries. The effect 
will be to reduce adjustments to Supply-Manufacturing items, while increasing 
adjustments elsewhere, resulting in output that is no longer a maximum likelihood 


estimate. 


The effects will be more complex in cases where more than one cell contributes non- 
zero values to a subtotal (e.g. a “services industries” subtotal in the above example) 
but in general, weighting derived cells such as subtotals has the effect of reducing 
adjustments to the cells that contribute to those subtotals, causing errors in the 
balancing. 


If all cells are affected equally, these errors may largely cancel out, but this will not 
always be the case. For example, in the table presented in 2.1, Use-Steel-Car 
Manufacturing contributes to four different totals (Total Use for product, Total Use for 
industry, Total Input for industry, Total Use for all products and industries) but Use- 
Iron Ore-Exports contributes to only three totals (no Total Input). This effectively 
over-weights the Car Manufacturing data at the expense of Exports data, leading to 
larger-than-optimal adjustments in Exports. 


Hence, when setting weights we should distinguish between basic and derived cells, 
and set zero weights for derived cells. (The weights on their contributing cells will still 
act to limit adjustments to derived cells.) 
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The second type of dependence can arise due to mechanisms such as sample rotation 
and selection methods. 


For example, the ABS’ Economic Activity Survey (EAS) is an important contributor to 
industry supply and use data. EAS is an annual survey, and the sample selection is 
designed with the objective that where possible, businesses selected for the survey 
will remain in sample for three years before rotating out. In cases where we’re 
balancing multiple years of data simultaneously, this could lead to correlation between 
sampling errors for the same industry and product. 


However, this problem is partly mitigated by EAS’ use of regression weighting that 
reduces the impact of sampling error, and previous ABS investigations have found that 
the sample rotation strategy causes only a small covariance in year-to-year estimates. 


Sampling issues can also cause covariance within a single year’s data. For example, a 
single chemical manufacturing classification may cover several different specialties, 
each with different inputs and outputs. This could lead to covariance between some 
supply and use items within this industry. 


In theory, given full data on variance and covariance of the epsilon terms, the MLE 
approach discussed above might be generalised to produce estimates based on the 
exact likelihood for a given estimate. However, this would be a very complex 
approach requiring a large number of parameters to be estimated.’ As discussed in 
Section 5 it is difficult to obtain good data even for the variance of individual items. 
Quantifying interactions would likely be considerably more difficult; without good 
quality inputs it is unlikely that the complexity of such an approach would be justified. 


As a general rule it seems reasonable to assume that such covariance between basic 
cells will be small relative to the variances of those cells, and hence can be ignored. 
Section 6 addresses an important exception, the case of balancing a time series where 
movement preservation is required. 


7 Fora single-year Supply-Use table with ~10,000 non-zero terms, the variance/covariance matrix would contain 
approximately 50,000,000 unique parameters. Even this would not be enough to calculate the likelihood for a 
given estimate, because pairwise covariances don’t contain enough information to fully define a multivariate 
distribution. 
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3.3 Modifications to weighting 


The MLE framework described above is based solely on the (estimated) error 
function. In some cases it may be appropriate to modify weights based on other 
factors. In particular, if one particular group of cells within the table is unusually 
important (e.g. crucial to policy-making) it might be desirable to improve the accuracy 
for these cells, even at the cost of reduced accuracy elsewhere in the table’. 


This can be achieved by fixing their values, increasing their weights, and/or restricting 
adjustments to these cells. However, excessively strict weighting/restriction may 
actually worsen accuracy for these cells. Balancing combines direct information about 
a cell (its unbalanced estimate) with indirect information (the rest of the unbalanced 
table); putting too much priority on the direct information risks losing accuracy 
through ignoring indirect information. Simulation studies may be required to 
determine where over-weighting becomes counter-productive. 


Even where weights are adjusted to prioritise specific parts of the table, it’s desirable 
to be transparent about the distinction between weighting based on source accuracy 
and weighting based on output priorities. This can be achieved by defining weights 
relative to the “pure MLE” values. Using MLE balancing for a first cut solution also 
provides a starting point for discussion about output priorities. 


8 In general, important data items will be prioritised in data collection, so we may expect that these will have 
high accuracy and hence large weights even under pure MLE weighting, but this depends on the nature of the 
inputs. 
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4. WEIGHTING STRATEGIES 


Under the restrictions discussed in Section 3, we achieve a maximum likelihood 
estimate by setting cell weights equal to (or proportional to) 1/variance. When 
estimates of variance are available (e.g. data item comes from a sample survey 
amenable to standard methods of variance estimation) this provides a straightforward 
rule for setting weights. 


However, many items in National Accounts tables come from other sources, e.g. 
administrative data, which do not provide error estimates. This requires a fall-back 
weighting strategy for use in such cases, i.e. a method for producing an approximate 
estimate of variance. 


Information that could be used for this purpose includes: 

° Magnitudes of unbalanced estimates; 

2 Expert judgement on accuracy; 

. Cell type e.g. tax data vs. supply vs. intermediate use, how collected. 


Since expert opinion is often based on cell type, these two may be considered 
together: e.g. subject matter experts may give an accuracy rating for each type of cell. 


A simple approach to weighting can be expressed as: 


Here w is the weight assigned to a cell, m is a function of cell magnitude (estimated 
from unbalanced magnitude, or other sources — generally a power of the magnitude), 
Tis a function of cell type/expert judgement, and R is a constant, with all terms 
positive. The product km,z, can be taken as an estimate of variance for cell 7. 


If all weights are set through this method, then the choice of Rk has no effect on the 
solution, and it can be set to 1. If some are set through other methods e.g. for data 
sources with known variance, then k can be used to calibrate the relative weights for 
these different methods. The methods used at Statistics Netherlands and the U.S. 
Bureau of Economic Analysis are along these lines, with some differences in the 
formulation (see Eurostat, 2013 and Chen, 2012). 
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4.1 Weighting by magnitude 


We considered two candidates for this term: 


Method 1: oe 


Method 2: mM, = 


Method 1 is equivalent to assuming that cells of the same type have similar coefficients 
of variance, with standard errors are in proportion to the magnitude of the cell. In the 
single-constraint balancing scenario discussed in 6.2, this results in adjustments 
proportional to the square of cell magnitude. 


Method 2 is equivalent to assuming that standard errors vary in proportion to the 
square-root of magnitude (hence, disregarding differences in cell type, larger cells 
have larger SEs but smaller RSEs). In the single-constraint scenario, this results in 
adjustments proportional to the unbalanced cell magnitude. If all cells have the same 
sign, this is effectively proration. 


Statistics Netherlands originally used Method 1 as discussed in Eurostat (2013) but 
now also uses Method 2 (Dr. Reinier Bikker, personal communication). The BEA has 
used both approaches in different parts of their balancing work (cf. Chen, 2012; Chen 
et al., 2014). Di Fonzo and Marini (2009) discuss both options and favour Method 1. 
Fortier and Quenneville (2009) also discuss both, and state that Statistics Canada’s 
TSRAKING reconciliation procedure uses Method 2. 


Some arguments in favour of Method 2: 


° In economic outputs, larger values tend to be of greater importance and so are 
collected with smaller RSEs. For example, ABS industrial collections prioritise 
sample allocation to give high accuracy for major industries such as mining. 


° In non-economic contexts, while smaller values may often be of great interest, 
these tend to be associated with larger RSEs. For example, in estimating counts 
of various subpopulations from a simple random sample, the numbers counted 
will follow a Poisson distribution with variance for each subpopulation 
proportional to its total numbers. 


° Proration is a familiar concept to many users of National Accounts data, and 
Method 2 can be understood as similar in effect to pro-ration. 


° Under the assumption of independent errors, if we aggregate two or more cells 
to form a single total, the variance of their sum should match the sum of their 
variances. Method 2 is consistent with this (assuming all aggregated items have 
the same sign, which would usually be the case); Method 1 is not. 
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° This is particularly important in cases where we need to balance the same data 
under different aggregations: e.g. a change to industry classifications may 
require producing tables for the same reference period under both old and new 
classifications, or considerations of problem size may require balancing a table at 
a broad classification before rebalancing at finer levels. In these cases, Method 2 
is likely to give better consistency between outputs; for the single-constraint 
scenario it gives perfect consistency. 


° Even in the single-constraint scenario, Method 1 can produce paradoxical results 
that greatly distort relativities between values in the same constraint group. 


0 Consider a table with 100 entries of unbalanced value $10m each, and one 
entry of unbalanced value $100m (i.e. unbalanced total $1100m), adjusted 
to match a total of $900m. Under Method 1, the small entries would each 
be adjusted by -10% to $9m, but the large entry would be adjusted by — 
100% to $0. Under Method 2, all entries are adjusted by —18%, preserving 
relative sizes. Fortier and Quenneville (2009) describe what appears to be 
a similar problem with simulated weekly data under Method 1. 


0 As well as being implausible, adjusting values to zero may increase 
computational difficulty of the problem by activating inequality constraints; 
a method that approximates pro-ration is much less likely to do this. 


However, given that Method 1 is commonly used, ABS should conduct further 
investigation to determine which method is preferable, e.g. by comparing behaviour 
of the two on test cases and by further research into the reasons behind other 
agencies’ choices. 


4.1.1 Graphical comparison of weighting methods 


As a preliminary investigation, our team experimented with balancing actual Australian 
Supply-Use tables using the two methods discussed above. 


This investigation used the 2010-2011 Supply-Use table, taking a version balanced by 
the current RAS-based system and also an unbalanced version from before the RAS 
process. Subject matter experts gave subjective reliability ratings for different groups 
of data within the unbalanced table: e.g. Intermediate Use items were typically rated 
as “40% reliable”, most Supply items were rated as “80% reliable”, and export/import 
and household and government final consumption expenditure items were rated as 
“90% reliable”.’ 


9 These ratings were made on an ad-hoc basis, to allow us to experiment with different balancing methods. They 
are not intended as a precise assessment of data quality for these different items and should not be interpreted 
as such. 
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Cells were then weighted at: 


1 
Method 1: Wi=a 
(1-1) 
1 
Method 2: W, = 


i a 
x; 


(1-1) 


where r, is the reliability rating discussed above: hence e.g. a cell with reliability rating 
of 0.9 would have twice the weight of another cell with reliability 0.8, if their 
unbalanced magnitudes are equal. (This is not the only way to translate such ratings 
into weights, and not necessarily the best; the object here was to explore the ability of 
WLS balancing to incorporate reliability measures, rather than to set those measures.) 
Reliability of 100% is interpreted as a non-adjustable cell. 


We then balanced the 2010-11 data via WLS, using Method 1 and 2 weights. For these 
balanced tables and for the RAS output, we calculated adjustments for each cell. 


Figure 4.1 shows a comparison of the adjustments for RAS and for WLS Method 1. 


4.1 Comparison of raw adjustments, RAS vs WLS Method 1 


Comparison of adjustments: RAS vs WLS Method 1, raw adjustments 
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10 
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In this plot, each dot represents a single cell in the table. The size of the dot indicates 
its unadjusted magnitude, the colour indicates its reliability rating, and its location 
indicates the adjustments it received: the x-coordinate shows the adjustment under 
RAS, and the y-coordinate shows the adjustment under WLS Method 1. Cells receiving 
no adjustment, and total/subtotal items, are omitted. 


Balancing outputs were rounded to multiples of $1m, with the result that many points 
would be plotted on top of one another; in these graphics those values have been 
randomly jittered by up to $0.4m to separate such points and help in visualisation. 
Both axes use a double-sided log scale. 


Points in the top and bottom quadrants of the plot are those where WLS Method 1 
produced larger adjustments than RAS, and vice versa for those in the left and right 
quadrants. 


Adjustments can also be represented as a proportion of unadjusted cell magnitudes: 


4.2 Comparison of proportional adjustments, RAS vs WLS Method 1 


Comparison of adjustments: RAS vs WLS Method 1, proportional adjustments 
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4.3 Comparison of proportional adjustments, RAS vs WLS Method 2 


Comparison of adjustments: RAS vs WLS Method 2, proportional adjustments 
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4.4 Comparison of proportional adjustments, WLS Method 1 vs Method 2 


Comparison of adjustments: WLS Methods 1 and 2, proportional adjustments 
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Some points of note: 


Large cells often get large adjustments in dollar terms, but in proportional terms 
their adjustments are usually small. 


Method 2 tends to make smaller adjustments to large cells, but larger 
adjustments to small cells, compared to Method 1 (as would be expected from 
theory) 


Individual cells may receive larger adjustments under WLS balancing, but overall 
WLS (both methods) tend to produce smaller adjustments than RAS, especially 
for those cells rated at 90-95% reliability. 


This is especially noticeable when looking at the largest adjustments: e.g. WLS 
rarely adjusts even a low reliability cell by more than ~20%, whereas such 
adjustments are much more common under RAS. 


WLS and RAS often adjust the same cell in different directions (i.e. points in top- 
left/bottom-right quadrants); the two WLS methods almost always adjust in the 
same direction, although the size of adjustment may be very different. 


Appendix A shows RAS vs WLS adjustments, split by cell reliability. In general, even at 


low reliability, WLS adjustments tend to be smaller than RAS. 


4.1.2 Special cases for magnitude 


In some cases, the unbalanced cell magnitude may not be a good proxy for variance. 


This is particularly a concern with “net” items that have been estimated as the 


difference of two large positive quantities. In this case, the standard error may be very 


large compared to non-”net” items of similar magnitude. 


Many of these “net” items will fall under the “dependent by definition”/’derived cells” 


case discussed in 3.2, in which case their weight can simply be set to zero. If this is 


not the case (e.g. the data used to derive the “net” item doesn’t appear separately in 


the table) then it may be necessary to consider a substitute for magnitude, e.g. choose 


a cell expected to have comparable magnitude to those contributing to the net. 


There may also be cases where a data item is believed to be small but non-zero, but 


has not been measured (e.g. for reasons of collection cost or respondent burden) and 


hence the unbalanced value is zero. Normally a zero input would be treated as non- 


adjustable (estimated zero standard error). In such cases, we might be willing to 


believe a small adjustment from zero—but in practice, it seems better to keep them 


fixed at zero, since we have no defensible basis for estimating the adjustment, and 


allowing a small adjustment will have little effect on the balancing. 
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4.2 Expert-guided weighting 


Even where no rigorous estimates of standard errors are available, National Accounts 
subject matter experts have a good understanding of the data sources feeding into 
these tables, and can provide qualitative information about their expected reliability. 
Although this introduces a subjective element to the weighting, it seems far better 
than ignoring this knowledge and basing weighting only on magnitude. 


Making use of this knowledge requires translating a qualitative assessment into a 
numerical estimate of relative accuracy. We consider three options, of the sort that 
have previously been used by Statistics Netherlands and the BEA: 


° Subject matter experts estimate the standard error for each type of cell, relative 
to other cell types (e.g. a supply cell has ~5 x the standard error of a tax cell of 
the same magnitude). 


: Subject matter experts estimate the magnitude of reasonable adjustments for 
each type of cell, relative to other cell types (e.g. we expect supply cells to 
receive ~10X the adjustment of tax cells for the same magnitude). 


° Subject matter experts rate cells on a simple scale (e.g. 1 = least reliable to 10 = 
most reliable). 


Ratings from the first of these methods can be translated to weights by setting 7, such 
that 1/7, varies in proportion to the square of the estimated SEs, relative to other cell 
types at the same magnitude. 


For the second method, the single-constraint scenario suggests that it would be 
reasonable to set z, such that 1/7, varies in proportion to the relative expected 
adjustment size. 


For the third method, Statistics Netherlands convert the integer rating to a weight by 
an exponential translation. 


The first two methods have the advantage of making a quantitative prediction about 
the data. These predictions could be compared to outcomes as a quality check, and 
can also be used to choose an appropriate value for Rk so that these cells are not over- 
or under-weighted relative to those whose SE is derived from other methods. 


However, the third method may be simpler to apply, and makes the subjective nature 
of the data more visible. 
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4.3 MLE weighting for unknown errors 


As an alternative to the methods discussed in Section 4.2, it may be possible to 
increase automation and reduce subjectivity of the weighting process by using 
maximum likelihood estimation to determine appropriate values for, . 


For example, suppose that the cells in our table for which standard errors are not 
available can be divided into sets A, B, and C, with the expectation that cells within the 
same sets will have similar levels of error. We then restrict 7, to the form: 


Kk, ifieA 
T,=4k,,ifieB 


Z 


Ko, iffeC 


for some parameters K,,K,,Kc- 


For any given values ofx,,«,, and«,, we can find the WLS/MLE solution for Xx , and 
calculate the likelihood associated with that particular estimate; denote this as 

I (K4,&p,&). We may then optimise over possible values of {«,,«,,-} to find the 
parameter choices that maximise that likelihood. 


In theory this method could be applied to generate new weights for every balancing 
problem. In practice, calculating weights is likely to be computationally intensive, and 
it may be desirable to keep weights stable from one reference period to the next, so it 
may be preferable to use the same set of weights for several years and update 
occasionally. 


If weights are instead calculated by the methods discussed in Section 4.2, the same 
MLE approach could be used to determine the constant k& that calibrates these weights 
against those derived from known standard errors. This would likely be less 
computationally demanding, since it only requires estimating one parameter. 


Since expert judgement would still be required to classify cells into similar groups 
prior to applying the MLE process, this approach should be seen as reducing 
subjectivity in weighting rather than completely eliminating it. 


4.4 Example of MLE method 


As an example of how this technique might be applied, we will return to the two 
weighting options discussed in Section 4.1. Recall that cells were weighted at: 


Ww. = oti a (Method 1) 
A (lar 


Ww. = oe (Method 2) 


: &|(1-7) 
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This corresponds to a variance model of: 


=k(1-n)|X, 


where @ equals 1 (for Method 2) or 2 (for Method 1) and k is some positive scalar. 


We would like some objective criteria to determine which of these two models is 
more appropriate. In the absence of other information, it seems reasonable to choose 
the one that gives the highest likelihood solution. 


Returning to the likelihood function derived in Section 3 and substituting in this 
estimate foro? , we get an expanded likelihood function: 


f (é,k,@,r) a 
PL (22ka-n)|&!) 
= ni{(3%,) exp(-2?w, T) ; 
1 
w, = ———— - 
© (en) si) 


Given sufficient data and computing power, we could perhaps optimise over all of 

( é,k, 0,1 ) and choose the values that maximise the resulting likelihood. However, to 
simplify this example, we will treat r as a fixed constant vector, and restrict 8 to 
either 1 or 2, with k and € as continuous. 


As in Section 3, maximising the likelihood is equivalent to minimising the negative log 
of that likelihood: 


OF =-In( f (&,k,4,r)) 


=E [atm ey )-3n( se 3) 


“Er [Sevime )+In(2 *)-In(0)] 


with k and @ now treated as variables within the optimisation, this is no longer a pure 
quadratic optimisation problem. It could still be solved through generic optimisation 
methods e.g. gradient descent, but this might be computationally expensive given the 
large number of variables involved. 
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However, in this case we can simplify the optimisation by observing that for any given 
choice of 8, r (which together define w) the optimal values of € are independent of 
k. This can be seen by rewriting the objective function as: 


OF = 5 Daim +5 Dlin(k) +In (2) -In(w,)) 


For fixed values of k, r, and 9 (hence also fixed w) this expression is minimised if and 
only if Ds &°w, is minimised, and the solution to that minimisation is independent 
of Rk. 


Hence, for any given 0,r we can find the optimal values for € as a quadratic 
optimisation, and only then do we need to solve for optimal R. 


With all other parameters now fixed, discarding constant terms shows that the optimal 
Rk will be that which minimises: 


n a2 
5[ BP +In(k)) =min(e) +2 Dra | 


Solving for d/dk=0 then shows that the optimum lies atk = > ew /n. 


In this case, with r fixed and @ restricted to only two possible values, we can reduce 
the problem to two quadratic optimisation runs, rather than a large non-quadratic 


optimisation. 


Doing this for the 2010-2011 Supply-Use data discussed in Section 4.1.1 gives the 


following values: 
Method 1: R=0.00047, OF = 5283.7 
Method 2: R=1.03, OF = 28705.6 


Recalling that the objective function is the negative log of likelihood, lower values for 
the OF indicate higher likelihood. In this case, if we were satisfied with the other 
assumptions used for the balancing (in particular, we assume that the parameters in r 
have been set to their best possible values) then this would imply that Method 1 is 
preferable. 


In practice, the values for r have not been set rigorously for this example, and it may 
be that optimising on r as well as on k and @ would lead to different conclusions. It 
may also be that allowing different values of @ for different types of inputs, and/or 
non-integer 0, will give a better fit to the behaviour of the inputs. Hence, this should 
be taken only as an example of how such decisions might be made. 
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5. BALANCING WITH MOVEMENTS 


Many economic tables are compiled annually or quarterly, and changes in estimates 
over time (“movements”) are of interest, often as much so as the point-in-time 
estimates (“levels”). When adjusting such data, it may be desirable to control the 
adjustments to movements. 


Benchmarking theory quantifies adjustment to movements via the Denton Additive 
First Differences (AFD) and Proportionate First Differences (PFD) measures: 


AFD, , = (me Fs) (4 ca) 


1, 


where x,, is the original estimate or indicator for cell 7 at time period ¢ and X,, is the 
adjusted value. 


Minimising squared AFD or PFD adjustments is a well-known method for 
benchmarking individual time series (cf. Dagum and Cholette, 2006). Eurostat (2013) 
describes a balancing approach based on a combination of the WLS level-preservation 
function given in Section 3 of this paper, and weighted squared AFD or PFD terms. 


This section will examine AFD/PFD-minimisation approaches within the same sort of 
MLE framework applied in Section 3. It will show that AFD and PFD-based movement- 
preservation objective functions correspond to maximum likelihood estimators for 
random walk error models (in the case of PFD, approximately random walk), and will 
offer a modification of such methods to improve forward-series estimation. 


It will also demonstrate that these correspondences are not additive: even though a 
MLE for white-noise additive errors corresponds to a level-preservation objective 
function, and a MLE for random-walk additive errors corresponds to an AFD objective 
function, the linear combination of AFD and level-preservation functions (as used e.g. 
in Eurostat) does not correspond to the MLE for a combination of white-noise and 
random walk additive errors. 


Hence, although the two approaches are equivalent for important simple cases, they 
are not equivalent in more general cases. This section will discuss the pros and cons 
of a “strict MLE” approach as compared to the Eurostat method. 
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5.1 MLE justification for AFD and PFD balancing 


Section 3 discussed a scenario where the unbalanced values are related to true values 
by independent, normally distributed errors: x, =x,+é,, with ¢,=N (0, o;) . In that 
case we established that a balanced estimate X can be calculated as a maximum 
likelihood estimator for x by minimising the weighted sum of squared adjustment. 


Now consider a scenario where instead of the error structure assumed in Section 3, 
the errors behave as an additive random walk (ARW): 


t 
Xie = Xi4 x aie : 
k=1 


Here the “innovations” 77,, are independent and normally distributed, with mean 


Zero: 
1, =N(0,p;,). 


Note that 7,, represents total estimator error at the start of the series, while 77, ,., 
represents only the additional error accrued between one period and the next. Hence 
we will assume that p;, > ;,,, and approximate 77,, as having a flat prior. 


By definition, for ¢ > 1, 


Hence, for any given set of balanced estimates x , the implied estimates for the 


innovations are: 


Because the innovations are independent, we can apply the same arguments used in 
Section 3 to show that a maximum likelihood estimate is obtained by selecting the 
values for X within the feasible region that minimise a modified objective function: 


7 2 
Fo Yay) 
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The flat-prior approximation for 7,, allows us to drop out the ¢=1 terms from the 
objective: 7,,/ 9,, = 0so can be ignored. (This requires using the fact that 7,, cannot 
grow very large without some other 77,, also growing very large, and hence causing a 
non-optimal objective that need not be considered.) 


This then gives: 


Fm = DE (4/, |} =LE ve He Fvs)Cr-Rl) 


i t>l i t>l 


= >»: >i v,,AFD, ,” 


i t>l 


: 2 
where movement weights v are set as v,, =1/ 9;,. 


Hence we have shown that an AFD-based least squares objective function is equivalent 
to the MLE estimator obtained from an additive random walk: 


OF uw = >, >, V;,AFD,, = 


i t>l 


t 
MLE . zg, = Xi + Mies Nit = N(0, oar Pi, > Lee Vit = Vp 
i,t 


k=1 


This approach corresponds to AFD-based benchmarking methods, and to the additive 
movement-preservation component of the objective function presented in Eurostat. 


In some cases, it may be more appropriate to assume that errors scale in proportion 
to the underlying estimate. The most straightforward form for such a model would be 
a proportional random walk (PRW): 


with 7,,=N (0, Pir) and independent, as before. This leads to: 


A 
Xi 


1 = as 
Xx. Xx 


However, this form is inconvenient for an optimisation approach because the 
resulting objective function would require nonlinear constraints and/or a non- 
quadratic objective function. 


32 ABS * QUADRATIC OPTIMISATION FOR TABLE BALANCING IN OFFICIAL STATISTICS * 1352.0.55.157 


ABS METHODOLOGY ADVISORY COMMITTEE * JUNE 2016 


To avoid this difficulty, benchmarking methods often use the PFD movement- 
preservation measure instead, which is linear in the decision variables and hence gives 
a quadratic objective function when squared. 


In fact, the PFD measure corresponds to a modified version of the proportional 
random walk model (M-PRW) presented above: 


cal 
t 
MLE: g, =x, [Sine , Hy HN (0, Pir): Pi > Pres Vig = Vp 
k=l 


it 


Note that under the assumption that p;, > > p,, (i.e. the probability of the random 
walk crossing zero is negligible) the recipro¢al random walk is itself approximately a 
random walk. This can be proved by using the approximation (1+ h)* ~1—-h for 
small hb. 


Hence a PFD-type least squares objective function seems a reasonable approximation 
for a MLE estimator with a proportional random walk error model, in the interests of 
keeping to a tractable objective function. 


5.2 Addition of trend term 


In some cases the estimation error may show a consistent trend over time. For 
example, our unbalanced estimator x,,may come from a source that only has partial 
coverage of the population of interest, with that coverage steadily increasing or 
decreasing from period to period. 


In such cases, the measures discussed above may incur avoidable bias in estimation, 
because part of the estimator movement that they preserve is in fact due to systematic 
error. 


In Brent, Stuckey, and Davidson (2015) we considered the problem in the context of 
adjusting a quarterly series to match annual benchmark totals (assumed to be exactly 
correct). Benchmarks help mitigate this error, since the net movement over long 
periods has to match the movement between benchmarks. However, it often 
happens that the most recent benchmark is not yet available and so the most recent 
estimates are not subject to this constraint. 


Figure 5.1 below shows a fictional illustration of this situation. Both the annual 
benchmarks and the unbalanced series are increasing smoothly over time, but there’s 
a steadily increasing discrepancy, suggesting a growing systematic error in the 
unbalanced estimator. In the benchmarked years, reconciling to benchmarks 
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eliminates this error, so the balanced estimate shows higher growth than the 
indicator. But for the last year in the series, because no benchmark constraint yet 
applies, an AFD objective function requires it to match the unbalanced movement 
exactly, resulting in a decline in growth that will probably need to be corrected when 
the next benchmark does become available. As long as the relationship between 
benchmarks and unbalanced estimates continues to follow this pattern, the same 
error will recur every year. 


5.1 Adjusting a quarterly series to an annual benchmark using AFD 
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In our previous benchmarking work we found that this kind of bias could be greatly 
reduced, without substantial increase in errors, by forecasting the growth of a 
benchmark indicator ratio from benchmarked to unbenchmarked periods — for 
instance, via a “random walk with drift” approach. This idea can be adapted to an 
optimisation framework by modifying the error models discussed previously. 


We can amend the additive RW model used to justify the AFD-based objective function 
by adding a trend term: 


t 
Xp = Xi + yt + Sit 
k=l 


where s, is the expected increase in error per period. In general this is not directly 
observable; instead we treat it as another unknown to be chosen/estimated as part of 
the optimisation. 
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With this modification, our estimate for the innovation terms becomes: 
a ( Xi ) = (4 a yes ) mer a 


If we assume flat priors for {s,', this then leads to a simple modification to the 
additive random walk/AFD-based objective function: 


OE iy cache = > > Vi ( AED,, ai 5; i P 


i t>l 


Although this approach increases the size of the problem by adding new variables, the 
increase is not large — only as much as adding another period’s worth of data. The 
fitted trend terms may also be useful as a quality-control measure for inputs; a large 
trend term may suggest e.g. that the coverage of the unbalanced estimator is changing 
rapidly. 


The proportional versions of these functions may be modified in a similar way'®. An 
error model of the form 


t -1 
Nit = [-s + > ns] Xi 
k=1 


leads to a modified PFD-type objective: 


OF y_ prey sirend = > Vit (PFD,, ae S; ‘i . 


i t>l 


When balancing over very short intervals, including a trend term may lead to 
overfitting-type problems; in these cases, it may be better to use the non-trend 
versions presented in Section 5.1 above. For very long intervals, it’s possible that the 
trend behaviour will change; in such cases a “rolling window” approach may be 
helpful (e.g. fix older estimates five years after their benchmarks become available, 
and apply these objective functions only for recent years.) 


10 The additive version represented trend as s,f.. The choice of —s,¢ for PFD is made to achieve consistency in 
resulting objective functions and in interpreting the sign of s, ; in both cases, positive values of s, imply that 
the unbalanced estimate is increasing relative to true value. 
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5.3 Combining movement- and level-preservation approaches 


In many cases the true error model may resemble some mix of the white-noise type 
errors considered in Section 3 (which led to an objective function based on 
minimising changes to levels) and the sort of auto-correlated errors considered in 
Sections 5.1 and 5.2 (which led to an objective function based on minimising changes 


to movements, modulo trend). 
Those error models can be combined, e.g.: 


t 
Xi, =X, + Sjb+ Dae +é,, 
k=l 


(additive RW + trend + white-noise transients), or 


21 
t 
Xi =X, [-si + Ss + o 


k=1 


(approximately proportional RW + trend + transients) 


with the same assumptions noted previously on distributions of € and 7, and adding 
the assumption that € and 77 are independent of one another. 


The problem then becomes choosing values fors, € and 7 in a way that minimises 


OE ia > iy bs U,M, eu aS WE, 


i t>1 t 


with weights v, w defined as before and with the implied X satisfying all relevant 
constraints. The balanced table x is then defined as a linear function of these 
decision variables; constraints can be defined in terms of * or of §, € and 7 as 
preferred. Weighting parameters might be set by the same MLE approach discussed 
in Section 4.3. 


Note that this approach increases the size of the optimisation problem, approximately 
doubling the number of free variables, since each x,, has corresponding ¢and 77. 


If appropriate, the error model could be extended further, e.g. to include weighted- 
average type terms, but this will further increase the size of the optimisation problem 
and the number of weighting parameters required, presenting a risk of overfitting. 
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An alternate approach, as presented in Eurostat, is to optimise on a linear 
combination of the objective functions used for these individual cases.'’ For example, 
one such form is: 


OF srp+ivt > >) > v,, AFD, + oy » W;, (a a eG y ‘ 
t 


i t>l Z 


movements levels 


Although each of the two components corresponds individually to a MLE estimator 
based on models discussed previously, the sum of those components does not 
correspond to an MLE based on the sum of the two error models. For example, 
consider the following problem where a single series with one missing benchmark 
must be adjusted for consistency with benchmarks: 


Year 
1 2 3 
Unbalanced estimate 10 20 30 
Benchmark value 110 - 130 


Here the correct values for year 1 and 3 are known exactly, but we still wish to use the 
unbalanced estimates to help choose an appropriate value for year 2. 


Under a strict MLE approach, summing white-noise and additive random walk error 
terms (without drift) gives the total error model: 


t 
Xit = Xi4 + ee + ery 
k=1 


Under the flat-prior assumption for77,, , the resulting objective function is: 


_ ~2 ~2 
OF siiiiid == ». ONY + > Wer, : 


t>1 t 


Assuming weights are non-negative, the unique optimal solution for this problem is to 
set 7,, =100 and set all other 7 and éterms to zero, ie. X =(1 10,120,130)’ . This 
satisfies constraints and achieves an objective of zero, which clearly cannot be 
improved on, and cannot be matched by any other choice of variables. Note that this 
solution remains optimal regardless of choice of weights, as long as they remain non- 
negative. 


11 The full objective function given in Eurostat contains options for both proportional and additive movement 
preservation, as well as level preservation and two additional terms not discussed here. It does not contain a 
trend term, but the movement-preservation components could easily be modified to include one. 
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However, if instead we generate an objective function by adding the WLS and AFD 
objectives, we have: 


OF = ¥0,,AFD,2 + ¥w,,(%,,-%,,) - 


t>1 t 


Since constraints require X,, =110,*,, =130 this simplifies to: 


2 2 
OF sep. = Vi2 (12 — 20 410-110) +v,, (130-30 + 20 -X,, } 
+ w,, (110-10) + w,,(%,—20) +w,,(130-30)" 


=(V2+Vis)(%2 -120) +W,, (%,2— 20)" +10000w, , +10000w, , 


The optimal solution for this objective is: 


_ _ 120(v,,+v,3)+20w,, 
x, =—). oo. 

V2 > V3 + Wi 2 
This solution varies with the weights and hence cannot be equivalent to the MLE 
solution, which does not. Thus it can be seen that in general, adding the objective 
functions derived from two error models does not yield an objective function that 
corresponds to the sum of those error models. 


This difference arises because the two methods have somewhat different purposes. 
Objectives such as OF,,,,,,,, ate based in the philosophy that the aim of balancing is to 
satisfy constraints while minimising change from unbalanced movements and levels 
(see e.g. discussion in Eurostat) and the form of these methods makes them 
appropriate for that purpose, with weighting to manage necessary trade-offs between 
those two aims. 


As presented here, the MLE method aims to give an estimate that is most likely to be 
true (subject to flat-prior approximations and simplifications as discussed previously), 
which is not quite the same thing as change minimisation. In the example shown 
above, a balanced table of (110,120,130) requires a large change to the middle value — 
but given the starting values of 110 and 130, and an indicator that shows smooth 
growth across the period, this seems like a very plausible choice. 


Figure 5.2 shows another comparison between the OF,,.,,,,, approach and a MLE 
based on the combined error model. In this scenario, an annual estimate series is 
supplemented by accurate benchmark data for years 1, 6, and 11; we wish to adjust 
the estimate to match these benchmarks. Two options are shown: OF,,,,,,,, with both 
components weighted equally, and the MLE approach. 


38 ABS * QUADRATIC OPTIMISATION FOR TABLE BALANCING IN OFFICIAL STATISTICS * 1352.0.55.157 


ABS METHODOLOGY ADVISORY COMMITTEE * JUNE 2016 


5.2 Comparing balancing methods, with five-year benchmark data 
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In years 1, 6, and 11 the solution is constrained to benchmark values, but in other 
years the level-preservation component of the additive objective function pulls 
estimates back towards the annual estimate. In between benchmarks, over years 3—4 
and 8-9, this gives a solution that closely matches both movements and levels for the 
annual estimate, but at the cost of spurious peaks at the benchmark years. 


A pure AFD objective function can be interpreted as accepting the possibility of 
systematic error and of random walk-type errors; under this objective a systematic 
correction is “free” and random walk corrections are cheap, making it easy to correct 
them. However, when a level-preservation function is added, all systematic and RW 
corrections are heavily penalised (since they require changes across much/all of the 
series, and each data point attracts a penalty) largely negating this feature of the AFD 
objective. 


For this particular scenario, the problem could be resolved by reducing the level- 
preservation weighting to zero. But this results in a simple AFD function that may not 
be appropriate to other cases where independent errors are large. 


By contrast, a MLE method based on combined (RW + independent) errors avoids 
these artefacts, without removing the capability to incorporate independent errors. 
It retains the ability to make “free” systematic corrections and cheap RW corrections, 
but doesn’t assume that these are the only kind of errors. The MLE solution shown 
above is not sensitive to weighting choices; no matter how strongly or weakly the 
random walk component is weighted against the independent component, the 
solution to this particular problem is virtually unchanged because the systematic 
correction remains free. 
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Each method has some advantages and disadvantages: 


° The Eurostat method may be easier to understand intuitively because it allows 
the balancing problem to be understood as a sum of “preserve movements” and 
“preserve levels”. However, this separation requires treating movements and 
levels as independent, when in fact movements are completely dependent on 
levels. 


° The MLE approach is likely to be more computationally expensive because it 
requires fitting more variables. 


° The MLE approach allows use of MLE methods to automatically fit weighting 
parameters; it is less obvious whether this method would be appropriate with 
the Eurostat objective. 


° The MLE approach may provide clearer insight into the nature of errors in the 
inputs, potentially guiding improvements to data collection. 


Based on these considerations, it seems desirable to evaluate the MLE approach 
further, and compare its behaviour to that of the Eurostat-style approach. 


5.4 Setting weights for movements 


As for the level-preservation balancing discussed in previous sections, applying these 
methods require setting weights, or equivalently, estimating variances o’ and p’. 


The Statistics Netherlands weighting strategy as published in Eurostat varies the 
weights for AFD movements proportional to the inverse square of unbalanced 


a\ 2 aed 
magnitude : v,, 0 1/ Ke . The exact value is also modified by categorical-type factors 


based on e.g. expected accuracy of unbalanced data, and by parameters that control 
the relative importance of movement preservation vs. level preservation vs. other 
considerations. 


Weights for PFD movements have no dependence on unbalanced magnitude, and are 
determined solely by categorical and relative importance considerations. 


Both of these approaches are consistent with an assumption that expected magnitude 
of errors will scale in proportion to the cell magnitude, which also matches the 
weighting for level adjustments. 


Movements at time ¢ are affected both by errors in levels at time ¢ and at /-1 hence a 
more exact approach might base movement weights on both IK 


and cal _ In 


practice, these magnitudes will generally be similar, so ignoring 


Xe and defining 


weights relative to a seems a reasonable simplification under this weighting 


strategy. 
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Under the alternate assumption that magnitude of errors scales in proportion to the 

square root of magnitude, weights for AFD movements should be set proportional to 
1/]%,, 
be proportional to Ls 


(modified by other factors, as above) and weights for PFD movements should 


(no reciprocal) because larger values are expected to have 
smaller proportionate errors, hence modifications to errors will be weighted more 
highly. 


For the “strict MLE” approach discussed in 6.3, weights on the white-noise error terms 
or 1/ 


ox KR (2 
may be set proportional to either 1/ IK. Xie 


with proportionality constants 7, , set through MLE methods as discussed in Section 5 
(i.e. classify cells into a small number of groups, assume 7,, equal for all cells within 
the same group, and then choose values that give the maximum likelihood solution). 


Random walk terms are likely to correspond to systematic errors, or to other issues 
that can be treated in the same way (e.g. the use of an indicator that correlates to the 
variable of interest, but is measured in different units). 


In general, it seems appropriate to parameterise these in a way that lets the magnitude 
of expected RW errors vary approximately in proportion to the magnitude of the 
estimates. (A lower order relationship implies that for series with very large 
magnitude, the proportion of systematic error approaches zero; a higher order 
relationship implies the same for series with very small magnitude. Neither of these 
seem desirable.) 


2 
Sais weld : Pes, 3 
For an additive RW error model, this implies Var (7; ..:) = Pi, =i |X] ie. 
of 
Viet = ~ [2 
i,t Xie 


for some proportionality constants, ¢,,to be chosen by the same approach as for 7, , 
above. 


For the approximate proportional RW model, this implies that Var (tics) should be 
independent of IK. 


, i.e. set 
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5.5 Time series shocks 


Time series may sometimes experience unusual shocks that require careful 
consideration in balancing and benchmarking. 


Some statistical methods rely on assumptions about the time series nature of the true 
series, e.g. ARIMA-type models. Violations of those assumptions may cause large 
errors. The MLE approach in this paper does not make any assumptions about 
smoothness or continuity of the true series.’ Hence, disturbances such as a level shift 
or trend break in the true series may not require any special treatment. 


The MLE approach does rely on assumptions about the error structure. Changes to 
the error structure are more significant — e.g. a change to data sources that causes a 
sudden large change in systematic error, or an increase/reduction in the variance of 
independent errors. Options for addressing these would include allowing a break in 
the error model (e.g. fitting different parameters before and after the change) and/or 
limiting the time period used for fitting parameters, so that model expectations are 
not overly influenced by obsolete information. 


12 Although it could be extended to do so, by using a model of the true series as a Bayesian prior. 
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6. DIAGNOSTICS FOR BALANCING 


This section discusses diagnostics that might be used for evaluating the results of an 
auto-balancing process. This discussion focuses on level-preservation balancing as 
introduced in Sections 3—4, but approaches presented here could be extended to the 
time series methods discussed in Section 5 by considering individual error-model 
components rather than just net adjustments. 


Diagnostics for the numerical optimisation itself are out of scope for this discussion; 
we expect that the optimisation will be performed by an off-the-shelf solver with its 
own diagnostics to identify problems within that process, so we will assume here that 
the balanced output X is indeed an optimal solution for the objective function and 
constraints as defined. 


Our primary interest here is in identifying unusual results that might signal a problem 
in the way the problem has been posed (e.g. bad inputs), flagging these for expert 
scrutiny, and interpreting those results. A secondary objective is to compare 
performance of two different balancing methods. 


A simple approach is to look for large adjustments (in dollar terms and/or relative to 
the cell’s unbalanced magnitude) but on its own this is unlikely to be adequate since it 
does not incorporate knowledge about expected variances. 


6.1 Important cases for diagnostics 


Some important cases to consider in diagnostics: 


: Cell weight is set too low (i.e. we have overestimated potential errors, which 
may result in larger adjustments than are appropriate for that cell). 


° Cell weight is set too high (e.g. we have underestimated the variance of 
estimates for that cell, or failed to correct for known bias before auto-balancing). 


° Cell weight is appropriate but some combination of inputs and constraints leads 
to poor behaviour (e.g. the matrix expression of the problem is ill-conditioned). 
Solver diagnostics may assist in identifying such cases. 


These cases might apply to a single cell, or across a group of related cells (e.g. we 
underweight all supply items for a single industry). 


In the first of the listed cases, the under-weighted cell/s will receive larger adjustments 
than they ought, while other cells closely related (via constraints) to the under- 
weighted cell/s will tend to receive smaller adjustments than they ought. 


In the second case, the effect is reversed. An over-weighted cell will receive smaller 
adjustments than it ought, and its “constraint neighbours” will receive somewhat 
larger adjustments than they ought. 
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In the third, a cell may receive large adjustments in response to a discrepancy 
elsewhere in the table. 


Without an independent source of information on input variances, detecting under- 
weighting for individual values is likely to be difficult. If we estimate that the standard 
error for an unbalanced cell is $100m and weight accordingly, but the final balancing 
outcome suggests a residual of only $10m for that cell, this 2¢ght indicate that the 
true standard error was smaller than $100m (i.e. the cell was under-weighted). But it’s 
also quite possible that the realised error just happened to be smaller than the 
standard error. 


For larger groups of cells it may be possible to detect a pattern of errors: e.g. if we 
estimate that a specific measure consistently has standard error $100m, but over 
several years it never requires an adjustment of more than $10m, this may suggest that 
the standard error is less than estimated and the weights for the corresponding 
variables should be increased. 


An important diagnostic in regression is /everage, measuring the influence of an 
individual data point on the fitted model. A similar approach might be applied here, 
e.g. by measuring how much a small change to the weight for a given cell changes 
estimates elsewhere in the table. 


6.2 Use of objective function as a diagnostic 


In assessing the results of balancing we may wish to examine the adjustments made, 
to gain insight into the errors that might be present in our inputs, along similar lines 
to the examination of residuals to evaluate a regression model. Unexpectedly large 
adjustments might indicate a problem with input data or problem specification, but 
defining “unexpectedly” requires some thought. 


As noted previously, the objective function provides a good starting point for 
evaluating adjustments. In many operations research problems, individual terms in 
the objective function translate directly to costs or profits for a single component of 
the overall work (e.g. power loss on a particular line) and therefore they are likely to 
be of interest in their own right as part of a cost breakdown. 


Here, the “cost” is the overall unlikelihood of the solution. Assuming weights w,are 
set proportional to 1/variance (excluding dependent cells, which are weighted at 
zero), then the adjustment on the 7-th cell can be considered to contribute a “cost” 
of (&,/9, y = &’w, to the objective function discussed in Section 3. (Recall that the 
standard deviation for €, iso, .) 


Hence, the cells that make the largest contribution to the objective function represent 
the adjustments that should be considered least likely. These make good candidates 
for scrutiny: e.g. we might choose the ten or twenty cells that make the largest 


44 ABS * QUADRATIC OPTIMISATION FOR TABLE BALANCING IN OFFICIAL STATISTICS * 1352.0.55.157 


ABS METHODOLOGY ADVISORY COMMITTEE * JUNE 2016 


contributions to the score. This can also be aggregated to a group-level diagnostic, 
e.g. for comparing entire rows within a table, which corresponds to the likelihood of 
that group’s adjustments. 


A complication here is that even though adjustments attempt to estimate errors, the 
expected distribution of adjustments will not match the distribution of errors. 
Consider the simple case of balancing a group of cells to meet a single constraint S on 
their sum, with no other constraints active. As noted by Stone et al. (1942), a 
Lagrange multiplier calculation shows that the optimal solution has all cell 
adjustments proportional to the inverse of their weights — hence adjustments will be 
in proportion to variance (as implied in the weighting) and mot to standard error. 


Although this paper does not attempt a rigorous proof for more complex scenarios, 
this case suggests that cells with the highest standard error will tend to receive “less 
likely” adjustments (higher (&, / o, y ). Using the contribution to objective function as 
a diagnostic may over-emphasise such cells while potentially overlooking problems 
with low error (high weight) cells. Hence we suggest the use of &w,as a second 
metric for detecting unusual adjustment patterns that might otherwise be masked by 
large weights; in the simple case mentioned above, all cells would score equally on 
this metric. We are not aware of this metric being used in previous work. 


Under either metric, a high score does not necessarily signify a problem; it may simply 
be part of a trade-off that accepts large adjustment to one cell as a trade-off for smaller 
adjustments elsewhere. Nevertheless it may be helpful to flag such adjustments so 

that users are aware of the trade-off, and to inform decisions for future data collection. 


Figures 6.1 and 6.2 show examples of these two metrics as applied to balancing on test 
data from a 2010-2011 Supply-Use table. In the unbalanced data one product had a 
large gap between supply and use, leading to large (proportional) adjustments for the 
relevant Use items (marked with red diamonds). However, because most of these had 
small initial values and hence large weights, these adjustments appear large but not 
exceptionally so when assessed by their contribution to the objective function 171, 
(figure 6.1). 


Plotting weighted adjustments ~w, (figure 6.2) makes it much more obvious that 
these cells are receiving unusually large adjustments (in the context of their weights), 
suggesting that this discrepancy might benefit from further attention; for example, we 
might consider the possibility of systematic underestimation for these items, or that 
the accuracy on the Supply side has been overestimated. 


In cases where a given cell or group of cells has a non-zero adjustment, increasing the 
weight for that cell will always increase the overall objective function, but can increase 
or decrease that cell’s contribution to the objective function (i.e. 77w,) and in the 
single-constraint case will decrease its proportional contribution to the objective 
function. 
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6.1 Adjustments scored by contribution to objective function 7; 1, 
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6.3 Inequality-condition diagnostics 


The main category of inequality conditions is that of sign constraints, e.g. items that 
cannot conceptually be negative. 


Such a constraint will be considered “satisfied” if the x-variable in question is greater 
than or equal to zero. However, even when a zero or near-zero value is conceptually 
possible, it may be implausible. For example, if the unbalanced estimates show a non- 
zero value for production of some good, it’s most likely because at least one business 
has reported producing that good, and so estimating a zero value for the entire 
economy would be hard to justify. 


These expectations could be enforced by modifying constraints (e.g. replace “not 
negative” constraints with “balanced value cannot be less than 50% of unbalanced 
value”), noting that over-zealous constraints may increase the risk of creating an 
unsolvable problem, or of forcing unrealistically large adjustments elsewhere. 


Whether or not this is done, a solution that lies near or on its inequality constraints 
may signal problems in some aspect of the inputs. 


Hence it may be desirable to flag non-zero values that have been adjusted to zero (or 
exactly matching some non-zero constraint), and possibly those that have been 
adjusted to less than e.g. 50% of their unbalanced magnitude. 


6.4 Summary of balancing diagnostics 
In summary of the above, diagnostics for a balancing run might include: 


° List of the top-N largest adjustments from unbalanced values (in dollar value) 
along with context on these adjustments: e.g. magnitude of adjustment as 
percentage of original value, magnitude of revisions to percentage growth, 
magnitude of discrepancies for relevant rows/columns, comparison to 
adjustments from recent years. 


° List of the top-N largest adjustments in proportion to the unbalanced value, with 
context as above, excluding those where a large proportional change would not 
be of concern (e.g. the unbalanced value is very small). 


. List of the top-N most “costly” adjustments as measured by the objective 
function 771w,. 


° List of the top-N most “costly” adjustments as measured by nw,. 


° List of rows, columns, and other constraint groups receiving unusually large 
adjustments (relative to other comparable entities) as measured by the objective 
function. 
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° List of inequality conditions that are exactly met (i.e. solution lies on boundary), 
excluding those that were already on the boundary before balancing—may be 
redundant with “large proportional adjustments” above. 


° List of cells that have been adjusted to less than e.g. 50% of unbalanced 
magnitude—again, may be redundant with “large proportional adjustments”. 


These lists can then be examined to identify any results that seem questionable. 


Section 4.1.1 shows examples of graphical diagnostics, as used to compare the 
behaviour of different balancing options. 


6.5 Corrections following diagnostics 


In some cases examination of diagnostics may identify problems, e.g. a table value that 
should have been corrected before auto-balancing, or has been given an incorrect 
weight. 


In such cases it may be appropriate to correct the inputs and rebalance the table. 
However, in order to keep the balancing process as objective as possible, it’s desirable 
that there should be clear guidelines in advance about when it’s appropriate to do 
this. 


In particular, cell weights should not be increased merely because the cells have 
received an unexpectedly large adjustment. Doing so risks biasing outputs to meet 
user expectations, and may also cause increased adjustments to other cells. In some 
cases, increasing weights will not substantially reduce the size of an adjustment (e.g. if 
constraints leave no other way to resolve a large discrepancy). In this case it may be 
appropriate to interpret the results as an indication that we have under-estimated 
variance for these cells, in which case weights should be decreased (which will not 
reduce the adjustments, but may prevent them from being flagged as anomalous). 


Other areas of ABS work require similar considerations about whether to adjust an 
estimation process after viewing its outputs, e.g. decisions on whether to outlier an 
atypical survey respondent. These areas have developed processes to govern such 
decisions and reduce the risk of user expectations biasing results; it may be 
appropriate to adapt such processes here. 
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7. NONLINEAR CONSTRAINTS 


Some balancing applications require the use of nonlinear constraints and/or non- 
quadratic objective functions. For example, balancing that incorporates past and 
present prices may require that the balanced data satisfy conditions such as 

i ate ee 

This requirement cannot be expressed without either introducing a nonlinear 
constraint, or reformulating to eliminate the nonlinear constraints at the cost of 
introducing a non-quadratic objective function. Optimisation programs exist that are 
capable of handling nonlinear constraints and/or non-quadratic objectives. However, 
increasing generalisation often comes with costs, e.g.: 


° Increased computing requirements; 
° Complex and less transparent problem specification; 


° Reduced choice in software options (ideally ABS balancing would use the same 
optimisation tools as other applications across the organisation). 


For these reasons it may be desirable to solve such problems using tools designed 
only for linear constraints and quadratic objective functions. Here we discuss some 
options for doing so. 


7.1 Approximating constraints/objectives 


If we expect that our unbalanced inputs are reasonably close to the true values (and 
hence that the balanced outputs will also be close) then we can construct linear 
approximations to the constraints and/or quadratic approximations to the objective 
function. 


For example, if adjustments are expected to be small relative to initial values, the 
requirement X,X, =X,x, can be approximated as: 


Where the adjustments are expected to be large, this method can be iterated. We can 
use the linear approximation above to calculate a first-run set of balanced estimates 
x'and then substitute in the estimates from X' to produce a more accurate set of 
constraints for a second balancing run: 

KX + XIX) XR = KORG + EUKS - RSX). 
For this second run, X' is treated as constant and X*is the vector of variables to be 
optimised. 
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In some cases, iterative approaches of this sort can fail to converge and may even 
diverge: consider e.g. the “overshoot” behaviour of Newton’s method when 


1/3 


attempting to find a zero for y=x’’’). Use of iterative balancing will require further 


consideration of this issue. 


In a case that requires a non-quadratic objective function, a similar approach can be 
used: we estimate some quadratic approximation to the OF, optimise for this 
approximation, then calculate derivatives of the exact OF at the approximate solution 
and use these to construct a new quadratic approximation, repeating as necessary. 


7.2 Penalty function methods 


As an alternative to “hard constraints” (enforced exactly), we can use “soft constraints” 
that allow some inexactness. This can be done by incorporating the constraint into 
the objective function by means of a penalty term. 


For example, Statistics Netherlands (see Eurostat) includes linearised soft ratio 
constraints via an objective function term of the form w(%, — dX; ) where d is the 
required ratio. Note that as x,and *, become small this term approaches zero even 
when x, /X, is not close to d, implying that this “constraint” becomes weaker for 
small values (unless weighted to counteract this). 


One option here is to solve with weak or zero-weighted soft constraints, check for 
constraint violations, and iteratively increase the weight for constraints that show large 
violations until the solution is within acceptable tolerances. 
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8. SCALING CONSIDERATIONS 


Balancing problems can be expanded almost indefinitely, to finer levels and additional 
dimensions of classification, more years of data, etc. etc. Sooner or later this will lead 
to problems that are too large to solve with available time and resources. 


Several companies provide powerful cloud-based “optimisation as a service” solutions, 
where problems can be supplied to a large cluster of servers optimised for such work. 
However, cloud solutions may not be appropriate where confidential data is involved. 


For such cases, a divide-and-conquer approach can be used: break the problem into 
smaller sub-problems that can be combined to give an approximate optimal solution. 
For example, Input-Output tables entail a similar balancing problem to Supply-Use, 
but at finer classifications resulting in more variables in the optimisation. If this makes 
the system too large to solve, one option might be to balance at Supply-Use level, then 
use the Supply-Use values as additional constraints on the Input-Output problem; this 
effectively reduces the number of independent variables in the optimisation, and 
depending on the solution algorithm may reduce the difficulty of the problem. 


For multi-year balancing problems, rebalancing/rebenchmarking can potentially cause 
changes to data many years in the past. Through appropriate choice of constraints 
the back series can be left open to adjustment, or older years can be fixed, as 
preferred. However, a long-running benchmarking system with many variables can 
result in a very large optimisation problem. 


One option for dealing with this is a “rolling window” approach: e.g. we balance the 
data for years 1-5, ignoring later years, then rebalance for years 1-6 but fixing year—1 
data to the values previously calculated, then rebalance for years 1-7 with years 1—2 
fixed, and so on. 


Another option for reconciling large groups of time series that have both temporal 
and contemporaneous constraints is to use a two-step procedure that first 
benchmarks each annual series individually, optimising movement preservation within 
that series, and then adjusts to achieve contemporaneous balancing within each low- 
frequency time period (e.g. a year). See Di Fonzo and Marini (2009) and Fortier and 
Quenneville (2009) for details on such methods and discussion of other work in this 
area. Cycling methods may also be used (Lenzen et al., 2012b). 
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9. CONCLUDING REMARKS 


Optimisation-based adjustment has great potential for improving both the quality of 
ABS economic statistics and the efficiency of their production. 


One of the major challenges in optimisation balancing for National Accounts and 
similar tables will be managing size and complexity. Some of this can be achieved 
through appropriate software tools (outside the scope of this paper) but this must 
also be considered on the theoretical side. 


Over-engineering for theoretical accuracy increases the difficulty of maintenance and 
can in fact become a risk to real-world data quality, since complex systems are harder 
to interpret, debug, and maintain. Hence, optimisation balancing/benchmarking 
systems should favour simple approaches that produce reasonable outputs, rather 
than aiming for theoretical perfection. 


A major objective of the ABS transformation program is to increase automation in the 
production of economic statistics. Other agencies have already used WLS adjustment 
methods for these purposes, but typically these methods still require subjective 
involvement in determining accuracy weights. This is likely to require considerable 
effort by subject matter experts and ongoing maintenance to ensure that the 
weighting models are still appropriate. 


The MLE approach discussed in this paper needs further exploration to confirm its 
viability, but it has several attractive features: 


° It offers an objective method for making weighting decisions, one which can 
largely be automated. 


° It clearly identifies assumptions about the structure of measurement errors, and 
allows those assumptions to be changed. 


° It is consistent with simple balancing/benchmarking methods already in use 
(WLS level-preservation or AFD/PFD movement-preservation). 


° It suggests that simply adding movement- and level-preservation objective 
functions together may not be the most appropriate way to handle a 
combination of correlated and uncorrelated errors, and offers a solution that can 
still be applied within a WLS framework. 


A possible next step would be to test this approach on realistic, synthesised data: for 
instance, start with a balanced SU table, perturb it under different error models, and 
then test how accurately the MLE method and other candidates recover the 
unperturbed table and estimate the parameters of the error model. This simulation 
approach can also be used for other investigations, e.g. testing sensitivity to normality 
assumptions. 
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For reasons related to the ABS transformation timetable, our work to date has focused 
on Supply-Use balancing, but we expect to extend these ideas to other economic 
outputs, and potentially to other ABS estimation processes with similar requirements. 
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APPENDIX 


A. DIAGNOSTIC PLOTS 
Plots in this section follow the form discussed in Section 4.1.1. 


Comparisons of proportional adjustments under RAS and under WLS Methods 1 and 
2, separated by cell reliability rating. 
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RAS vs WLS Method 1, proportional adjustments, 40% cells 
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RAS vs WLS Method 1, proportional adjustments, 60% cells 
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RAS vs WLS Method 1, proportional adjustments, 80% cells 
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RAS vs WLS Method 2, proportional adjustments, 80% cells 
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RAS vs WLS Method 1, proportional adjustments, 90% cells 
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RAS vs WLS Method 1, proportional adjustments, 95% cells 
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RAS vs WLS Method 2, proportional adjustments, 95% cells 
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FOR MORE INFORMATION ... 


INTERNET 


LIBRARY 


www.abs.gov.au The ABS website is the best place for data 
from our publications and information about the ABS. 


A range of ABS publications are available from public and tertiary 
libraries Australia wide. Contact your nearest library to determine 
whether it has the ABS statistics you require, or visit our website 

for a list of libraries. 


INFORMATION AND REFERRAL SERVICE 


PHONE 


EMAIL 


FAX 


POST 


Our consultants can help you access the full range of information 
published by the ABS that is available free 

of charge from our website, or purchase a hard copy publication. 
Information tailored to your needs can also be requested as a 
‘user pays' service. Specialists are on hand to help you with 
analytical or methodological advice. 


1300 135 070 
client.services@abs.gov.au 
1300 135 211 


Client Services, ABS, GPO Box 796, Sydney NSW 2001 


FREE ACCESS TO STATISTICS 


WEB ADDRESS 


All statistics on the ABS website can be downloaded free of 
charge. 


www.abs.gov.au 
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